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Abstract 

Most  practical  partial-order  planning  systems 
employ  some  form  of  goal  protection.  How¬ 
ever,  it  is  not  clear  from  previous  work  what 
the  tradeoffs  are  between  the  different  goal- 
protection  strategies.  Is  it  better  to  protect 
against  all  threats  to  a  subgoal,  some  threats, 
or  no  threats  at  all?  In  this  paper,  we  con¬ 
sider  three  well-known  planning  algorithms, 

SNLP,  NONUN,  and  twbak.  Each  algorithm 
makes  use  of  a  different  goal-protection  strat¬ 
egy.  Through  a  comparison  of  the  three  al¬ 
gorithms,  we  provide  a  detailed  analysis  of 
different  goal  protection  methods,  in  order  to 
identify  the  factors  that  determine  the  per¬ 
formance  of  the  systems.  The  analysis  clearly 
shows  that  the  relative  performance  of  the 
different  goal-protection  methods  used  by  the 
systems,  depends  on  the  characteristics  of  the 
problems  being  solved.  One  of  the  main  de¬ 
termining  factors  of  performance  is  the  ratio 
of  the  number  of  negative  threats  to  the  num¬ 
ber  of  positive  threats.  We  present  an  artifi¬ 
cial  domain  where  we  can  control  this  ratio 
and  show  that  in  fact  the  planners  show  radi¬ 
cally  different  performance  as  the  ratio  is  var¬ 
ied.  The  implication  of  this  result  for  some¬ 
one  implementing  a  planning  system  is  that 
the  most  appropriate  algorithm  will  depend 
on  the  types  of  problems  to  be  solved  by  the 
planner. 

1  Introduction 

There  has  been  a  great  deal  of  work  recently  on  com¬ 
paring  total  and  partial  order  planning  systems  [Barrett 
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and  Weld,  1992;  Minton  et  &i.  1991],  but  little  has  been 
done  in  comparing  different  partial  order  planners  them¬ 
selves.  There  are  a  variety  of  design  decisions  that  must 
be  made  in  order  to  build  a  general  planner.  This  pa¬ 
per  focuses  on  one  of  these  design  choices  -  the  choice 
of  a  protection  strategy.  In  particular,  we  compare  the 
protection  strategy  employed  in  three  basic  planning  al¬ 
gorithms,  SNLP,  NONLIN  and  TWEAK. 

On  the  surface,  the  three  planners  are  quite  different. 
However,  on  a  careful  examination  one  can  finu  they 
mainly  differ  in  which  conditions  they  protect.  During 
planning,  an  inserted  plan  step  can  interact  with  previ¬ 
ously  inserted  steps.  If  a  goal  is  achieved  by  one  plan 
step,  then  later  it  conld  be  threatened  by  other  steps. 
A  goal  is  protected  by  removing  all  threats  by  impos¬ 
ing  additional  constraints  on  a  plan  whenever  a  threat 
is  detected.  Among  the  three  planners,  tweak  protects 
nothing,  NONLIN  protects  against  all  negative  threats, 
and  SNLP  protects  against  both  negative  and  positive 
threats. 

The  use  of  goal  protection  in  snlp  prevents  the  plan¬ 
ner  from  generating  redundant  plans  and  thereby  could 
potentially  reduce  the  sise  of  the  search  space.  However, 
enforcing  the  goal  protection  has  a  cost.  In  this  paper, 
we  show  that  none  of  the  planners  is  always  a  winner.  In 
some  domains  our  planner  based  on  tweak  greatly  out¬ 
performs  both  a  planner  based  on  nonlin  1  and  snlp. 
In  other  domains,  SNLP  and  nonlin  perform  much  bet¬ 
ter  than  tweak.  The  challenge  is  to  identify  the  fea¬ 
tures  of  the  domains  where  each  planner  is  expected  to 
perform  well,  so  that  practitioners  can  balance  the  pro¬ 
tection  methods  based  on  the  application  domain. 

In  the  following  sections,  we  first  review  the  three  al¬ 
gorithms.  Then  we  present  an  analysis  of  the  algorithms 
to  identify  their  relative  merits.  We  also  report  on  two 
critical  domain  features  that  have  the  greatest  impact 
on  the  performance  of  the  planners.  Finally,  we  present 
empirical  results  on  an  artificial  domain  to  support  the 
analysis. 


'For  ronvenience  we  will  simply  refer  to  them  as  twbak 
and  nonlin. 


2  Comparison  of  the  Algorithms 

This  section  presents  the  SNLP,  TWEAK,  and  NONLIN 
planning  algorithms.  First,  we  present  the  snlp  algo¬ 
rithm  based  on  the  algorithm  descriptions  of  McAllester 
and  Rosenblitt’s  Find-Completion  algorithm  [McAllester 
and  Rosenblitt,  1991]  and  Barrett  and  Weld’s  POCL  al¬ 
gorithm  [Barrett  and  Weld,  1992].  We  start  with  this 
algorithm  because  we  can  build  on  the  elegant  algo¬ 
rithm  description  and  implementation  provided  in  pre¬ 
vious  work.  Then,  we  describe  the  changes  necessary 
to  transform  the  SNLP  algorithm  into  algorithms  that 
implement  NONLIN  [Tate,  1977]  and  tweak  [Chapman 
1987]. 

2.1  The  SNLP  Algorithm 

In  the  planning  algorithms  that  we  consider  below,  we 
follow  the  notations  used  by  Barrett  and  Weld  [Barrett 
and  Weld,  1992],  A  plan  is  a  3-tuple,  represented  as 
(S,0,B),  where  5  is  a  number  of  steps,  O  is  a  set  of 
ordering  constraints,  and  B  the  set  of  variable  binding 
constraints  associated  with  a  plan.  A  step  consists  of  a 
set  of  preconditions,  an  add  list,  and  a  delete  list.  The 
binding  constraints  specify  whether  two  variables  can  be 
bound  to  the  same  constant  or  not. 

The  core  of  SNLP  is  the  recording  of  the  causal  links 
for  why  a  step  is  introduced  into  a  plan,  and  for  protect¬ 
ing  that  purpose.  If  a  step  Si  adds  a  proposition  p  to 
satisfy  a  precondition  of  step  Sj ,  then  5<  A  Sj  denotes 
the  causal  link.  An  operator  5*  is  a  threat  to  Si  Sj  if 
Sk  can  possibly  add  or  delete  a  literal  q  that  can  possi¬ 
bly  be  bound  to  p.  For  convenience,  we  also  refer  to  the 
pair  (Sk,  Si  Sj)  as  a  threat.  In  addition,  we  define  an 

operator  Sk  to  be  a  positive  threat  to  Si  Sj,  if  Sk  can 
possibly  be  between  Si  and  Sj,  and  Sk  adds  a  literal  q 
that  can  possibly  be  bound  to  p.  Likewise,  Sk  is  a  neg¬ 
ative  threat  if  it  can  possibly  be  between  Si  and  Sj,  and 
deletes  a  literal  q  that  can  possibly  be  bound  to  p. 

The  following  algorithm  which  is  an  adaptation  of 
McAllester  and  Rosenblitt’s  Find-Completion  algorithm 
[McAllester  and  Rosenblitt,  1991]  and  Barrett  and 
Weld’s  POCL  algorithm  [Barrett  and  Weld,  1992],  has 
been  shown  to  be  sound,  complete,  and  systematic 
(never  generates  redundant  plans).  Let  the  notation 
codesignate (R)  denote  the  codesignation  constraints  im¬ 
posed  on  a  set  of  variable  pairs  R.  For  example,  if 
R  =  {(z«ilfc)  !  *  =  1,2,.. -fc},  then  codesignate (R)  = 
{xj  ss  yi  |  t  s=  1,2, ...As}.  Similarly,  noncodesignate (R) 
denotes  the  set  of  non-codesignation  constraints  on  a 
set  R  of  variable  pairs.  The  parameters  of  the  algorithm 
are:  S=Steps,  0= Ordering  constraints,  B=  Binding  con¬ 
straints,  G—  Goals,  T=  Threats,  and  L= Causal  links. 

Algorithm  snlp((S,  O,  B),T,  G,  L ) 

1.  Termination:  If  G  and  T  are  empty,  report  suc¬ 
cess  and  stop. 

2.  Declobbering:  A  step  Sk  threatens  a  causal  link 
«i  A  sj  when  it  occurs  between  Si  and  Sj ,  and  it 
adds  or  deletes  p.  If  there  exists  a  threat  t  (E  T  such 
that  t  is  a  threat  between  a  step  s»  and  a  causal 
link  Si  A  Sj  €  L,  then: 


•  Remove  the  threat  by  adding  ordering  con¬ 
straints  and/or  binding  constraints  using  pro¬ 
motion,  demotion,  or  separation.  For  com¬ 
pleteness,  all  ways  of  resolving  the  threat 
must  be  considered. 

-  Promotion:  O'  =  0|J{** -<*«}.  B'  =  B 

-  Demotion:  O'  =  0[J{4y  -<«*},  B'  =  B 

—  Separation: 

O'  -  0(J{si<Sk}\J{sk<8j}.  Let  q  be 
the  effect  of  sk  that  threatens  -p  and  let 
P  be  the  set  of  binding  pairs  between 
q  and  p.  B'  =  B(J<r,  where  c  6  {«  | 
a  =  noncodesignate  (*)(Jcodesignat  e  ( P  — 
s),  where  s  C  P  A  s  ^  0}.2 

•  Recursive  invocation: 

SNLP ((S,  O',  £'),  T  -  {<},  G,  L ) 

3.  Goal  selection:  Let  p  be  a  proposition  in  G,  and 
let  Snee<j  be  the  step  for  which  p  is  a  precondition. 

4.  Operator  selection:  Let  S-jj  be  an  existing 

step,  or  some  new  step,  that  adds  p  before  S_mj. 
If  no  such  step  exists  or  can  be  added  then  back¬ 
track.  Let  V  =  IU^Cadd -^need).  ^  = 
SU{Sadd}.°'  =  <?U{Sadd^need}.  and  B>  = 
£(J  the  set  of  variable  bindings  to  make  add 
p.  Finally,  update  the  goal  set:  G'  —  (G  —  {p})|J 
preconditions  of  if  new.  For  completeness, 

all  ways  of  achieving  the  step  must  be  considered. 

5.  Threat  identification:  Let  T  —  {t  |  for  every 
step  Sk  that  is  a  positive  or  negative  threat  to  a 
causal  link  -£»  Sj  6  L',t  —  (a*,  Si  Sy)}. 

6.  Recursive  invocation: 

StiLP((S',0,,B,),T',G',L,). 

2.2  The  NONLIN  Algorithm 

snlp  is  a  descendant  of  nonlin  [Tate,  1977],  so  the  al¬ 
gorithms  are  quite  similar  and  differ  mainly  in  which 
threats  they  protect  against  and  how  they  perform  sep¬ 
aration.  These  two  differences  stem  from  the  added  con¬ 
straints  on  SNLP  that  are  used  to  ensure  systematicity. 
NONLIN  also  provides  some  additional  capabilities  such 
as  hierarchical  task-network  decomposition,  but  these 
capabilities  are  orthogonal  to  the  point  of  this  paper  and 
are  not  considered. 

The  first  change  to  the  SNLP  algorithm  is  in  the  threat 
identification  step.  In  contrast  to  SNLP,  only  the  negative 
threats  are  added  to  the  list  T4: 

Threat  identification:  Let  T'  =  {<  |  for  every  step 
Sk  that  is  a  negative  threat  to  a  causal  link  s<  -£*  Sj  €  V, 
t  =  ( sk,Si  4  Sj)}. 

The  second  change  is  that  to  perform  separation, 
there  is  no  requirement  that  promotion,  demotion  and 
separation  are  made  mutually  exclusive.  In  this  case, 
separation  simply  entails  that  one  or  more  of  the  possi¬ 
ble  bindings  are  forced  not  to  codesignate,  but  imposes 
no  ordering  constraints. 

3 The  possible  binding  constraints  are  mutually  exclusive, 
since  systematicity  requires  that  the  search  space  is  parti¬ 
tioned  into  non-overlapping  parts. 


Separation:  O'  =  0.  Let  q  be  the  effect  of  ** 
that  possibly  codesignates  with  p  and  let  P  be  the  set 
of  binding  pairs  between  q  and  p.  B'  =  where 

a  €  {<*  |  a  =  noncodesignate(e),  where  e  €  P}. 

As  we  will  see  in  the  experimental  results  section,  the 
differences  in  performance  of  goal  protection  methods 
employed  by  SNLP  and  NONUN  are  relatively  minor. 

2.S  The  TWEAK  Algorithm 

The  primary  difference  between  TWEAK  and  the  two 
previous  algorithms  is  that  instead  of  building  explicit 
causal  links  for  each  condition  established  by  the  plan¬ 
ner,  tweak  uses  what  is  called  the  Modal  Truth  Crite¬ 
rion  [Chapman  1987]  to  check  the  truth  of  each  precondi¬ 
tion  in  the  plan.  This  difference  results  in  four  changes 
from  the  snlp  algorithm  and  only  three  changes  from 
the  nonlin  algorithm.  The  differences  are  in  termina¬ 
tion,  separation,  goal  selection,  and  threat  identification. 
Each  of  these  are  discussed  in  turn. 

Since  tweak  does  not  maintain  explicit  causal  links 
for  each  precondition,  it  must  test  the  truth  of  all  of  the 
preconditions  in  the  plan  to  determine  when  the  plan 
is  complete.  It  does  this  using  the  Modal  Truth  Crite¬ 
rion  check  [Chapman  1987].  This  algorithm  takes  0(n3) 
time,  as  compared  with  the  0(1)  time  termination  rou¬ 
tine  of  SNLP.  We  will  refer  to  the  algorithm  that  imple¬ 
ments  the  Modal  Truth  Criterion  as  mtc.  This  algorithm 
returns  true  if  a  given  plan  is  complete  and  otherwise  re¬ 
turns  a  precondition  of  some  step  in  the  plan  that  does 
not  necessarily  hold. 

Termination:  If  mtc ((S,0,B))  is  true,  report  suc¬ 
cess  and  stop. 

Similar  to  nonlin,  there  is  no  requirement  that  all  of 
the  separation  constraints  are  mutually  exclusive.  Thus, 
TWEAK  uses  the  same  method  for  separation  as  NONLIN. 

Separation:  O'  =  O.  Let  q  be  the  effect  of  ** 
that  possibly  codesignates  with  p  and  let  P  be  the  set 
of  binding  pairs  between  q  and  p.  B'  =  B\J<r,  where 
<r  6  {«  |  o  =  noncodesignate (e) ,  where  e  €  P}. 

Since  TWEAK  does  not  maintain  an  explicit  set  of 
causal  links,  there  is  no  explicit  record  of  which  pre¬ 
conditions  much  be  achieved.  Thus,  goal  section  is  done 
using  the  ate  algorithm.  The  mtc  returns  a  precondition 
of  a  step  in  the  plan  that  is  not  necessarily  true. 

Goal  Selection:  Let  p  be  the  precondition  of  step 
^need  retunie<i  by  the  mtc  procedure. 

Finally,  unlike  both  SNLP  and  NONLIN,  TWEAK  makes 
no  attempt  to  protect  all  of  the  previously  established 
preconditions  against  either  negative  or  positive  threats. 
TWEAK  does,  however,  ensure  that  at  each  step  all  neg¬ 
ative  threats  to  the  most  recently  built  causal  link  are 
removed.  However,  after  a  precondition  is  established 
and  threats  are  removed,  it  can  be  clobbered  again.  In 
such  a  case,  TWEAK  will  have  to  re-establish  the  condi- 
tion. 

Threat  identification:  Let  2nftt  =  -£•  Snee< j , 

which  is  the  causal  link  constructed  in  step  4.  Let  T*  = 
(t|for  every  step  i*  that  is  a  negative  threat  to 
f  = 

As  we  stated  above,  the  mtc  routine  for  the  termina¬ 
tion  check  is  more  expensive  than  that  for  SNLP.  How¬ 


ever,  this  does  not  mean  that  tweak  is  less  efficient 
than  SNLP,  since  in  many  cases,  TWEAK  will  explore 
fewer  nodes.  In  the  next  section,  we  consider  the  ma¬ 
jor  factors  that  affect  the  search  space,  and  present  a 
complexity  analysis  of  the  three  algorithms. 

3  Analyzing  the  Algorithms 

3.1  Algorithm  Complexities 

Let  e6  be  the  effective  branching  factor  and  ed  the  ef¬ 
fective  depth  of  the  search  tree.  In  both  algorithms,  eb 
is  the  maximum  number  of  successor  plans  generated  ei¬ 
ther  after  step  2,  or  after  step  5,  while  ed  is  the  maximum 
number  of  plan  expansions  in  the  search  tree  from  the 
initial  plan  state  to  the  solution  plan  state.  Then  with 
a  breadth-first  search,  the  time  complexity  of  search  is 

0(ebtd  *  Tnode), 

where  Tno<f«  is  the  amount  of  time  spent  per  node. 

We  next  analyse  the  complexity  of  the  algorithms  by 
fleshing  out  the  parameters  eb,  ed  and  Tnoif  In  this 
analysis,  let  P  denote  the  maximum  number  of  precon¬ 
ditions  or  effects  for  a  single  step,  let  N  denote  the  total 
number  of  operators  in  an  optimal  solution  plan,  and  let 
A  be  either  the  snlp,  nonlin,  or  tweak  algorithm. 

To  expand  the  effective  branching  factor  eb,  we  first 
define  the  following  additional  parameters.  We  use 
bjnew  for  the  number  of  new  operators  found  by  step 

4  for  achieving  p,  buold  for  the  number  of  existing  op¬ 
erators  found  by  step  4  for  achieving  p,  and  rt  for  the 
number  of  alternative  constraints  to  remove  one  threat. 
The  effective  branching  factor  of  search  by  either  algo¬ 
rithm  is  then 

eh  =  max{(  bjnew  +  buoldA),rtjl}, 

since  each  time  the  main  routine  is  followed,  either  step  2 
is  executed  for  removing  threats,  or  step  3  -6  is  executed 
to  build  causal  links.  If  step  2  is  executed,  successor 
states  are  generated,  but  otherwise,  (  bjnew  +  bMd) 
successor  plan  states  are  generated. 

Next,  we  expand  the  effective  depth  ed.  In  the  solu¬ 
tion  plan,  there  are  N  *  P  number  of  (p,Sneej)  pairs, 
where  p  is  a  precondition  for  step  5peed’  ^et  7x  be  the 
fraction  of  the  N  *  P  pairs  chosen  by  step  3.  For  each 
pair  (p,  Snee<j)  chosen  by  step  3,  step  5  accumulates  a 
set  of  threats  to  remove.  Let  tA  be  the  number  of  threats 
generated  by  step  5.  Finally,  let  v  be  the  total  number 
of  times  any  fixed  pair  (p,  £'nee<j)  is  chosen  by  step  3. 
Then  we  have 

edx  =  /a  *  N  *  P  *  tx  *  vA. 

A  summary  of  the  parameters  can  be  found  in  Table  3.1. 

For  SNLP,  each  pair  (p,  5nee^)  must  be  visited  exactly 
once.  Therefore,  ip  =  1  and  v,nip  =  1.  Also,  SNLP 
examines  every  causal  link  in  the  current  plan  in  step 
4.  Thus,  in  the  average  case,  the  amount  of  time  per 
node  is  half  of  the  total  number  of  links  in  the  solution 
plan,  i.e.,  N  *  P/2.  Thus,  the  average  time  complexity 
for  SNLP  is: 

0(max(  bjnew  +  bjold,nip,  ^t.n,r)N‘P*t"',r  *  N  *  P). 


Table  1:  Parameters  used  in  complexity  analysis. 


NONLlN’s  behaviour  is  similar  to  SNLP  in  that  each 
pair  (p,  Snee<j)  must  be  visited  exactly  once.  Therefore, 
fnonlin  —  1  and  vnmiin  =  1-  Also  similar  to  SNLP,  NON- 
LIN  examines  every  causal  link  in  the  current  plan  in 
step  4.  The  difference  between  NONLIN  and  SNLP  is  that 
NONLIN  resolves  only  negative  threats.  This  means  that 
in  general  nonlin  will  have  a  smaller  t  value.  The  aver¬ 
age  time  complexity  for  nonlin  is: 


0(m ax(  bjnew+  bjoldn^un,  rtn 


‘“vATvP) 


In  TWBAK,  ftVeak  <  1,  and  can  be  much  smaller  than 
one  since  tweak  does  not  build  explicit  causal  links  for 
every  precondition.  If  many  preconditions  already  hold, 
then  the  number  of  chosen  preconditions  by  step  3  in 
TWEAK  could  be  much  smaller  than  the  total  number  of 
preconditions  in  the  solution  plan.  Since  TWEAK  does 
not  protect  any  past  causal  links,  a  precondition  can  be 
visited  twice.  Therefore,  vtv,,ak  >  1-  Uvea k,  on  the  other 
hand,  should  be  much  smaller  than  t,nip  and  tn<miin, 
since  TWEAK  only  declobbers  for  the  most  recently  con¬ 
structed  causal  link,  and  only  negative  threats  are  con¬ 
sidered.  Thus  the  number  of  threats  is  much  smaller. 
Finally,  tweak  uses  MTC  to  check  the  correctness  of  a 
plan,  resulting  a  complexity  per  node  to  be  0((N  *  P)8). 
Overall,  the  complexity  of  tweak  is: 

0(max(  b-new  +  Jra  *  Ttv.ak 

where  to  =  /tweak  *N*P* t tweak  *  1 1, weak  and  Ttv.ak  = 

( N*P )8. 

In  the  next  section,  we  discuss  how  these  parameters 
change  with  certain  domain  features. 

3.2  Systematicity 

SNLP  is  systematic,  which  means  that  no  redundant  plans 
are  generated  in  the  search  space.  In  contrast,  neither 
TWEAK  nor  nonlin  are  systematic.  However,  a  plan¬ 
ner  that  is  systematic  is  not  necessarily  more  efficient. 
The  systematicity  property  reduces  the  branching  factor 
by  avoiding  redundant  plans.  However,  systematicity  is 
achieved  in  snlp  by  protecting  against  both  the  negative 
and  positive  threats,  which  increases  the  factor  t,  a  mul¬ 
tiplicative  factor  in  the  exponent.  Thus,  SNLP  reduces 
the  branching  factor  at  a  price  of  increasing  the  depth 
of  search.  Therefore,  one  can  get  a  systematic,  but  less 
efficient  planning  system. 


4  Domain  Features  and  Search 
Performance 

The  analysis  in  the  previous  section  can  be  used  to  pre¬ 
dict  the  relative  performance  of  the  three  planning  al¬ 
gorithms  in  different  types  of  domains.  An  important 
feature  of  a  domain  that  determines  the  relative  per¬ 
formance  of  any  two  algorithms  is  the  ratio  between 
the  number  of  positive  threats  and  number  of  negative 
threats.  The  ratio  is  an  important  factor  in  differentiat¬ 
ing  the  algorithms  because  the  major  difference  between 
any  two  algorithms  is  the  way  they  handle  positive  and 
negative  threats.  Among  the  three  algorithms,  TWEAK 
only  avoids  some  negative  threats,  snlp  protects  against 
ail  positive  and  negative  threats,  and  nonlin  protects 
against  all  negative  threats  but  not  the  positive  ones. 

4.1  Predictions 

The  major  difference  between  the  algorithms  manifest 
themselves  in  the  execution  of  Step  1,  the  termination 
subroutine,  and  Step  4,  threat  detection.  To  see  their  ef¬ 
fect  on  search  efficiency,  let  t+  denote  the  average  num¬ 
ber  of  positive  threats,  and  let  t  _  be  the  average  number 
of  negative  threats  detected  by  Step  4  of  snlp.  Let  R 
denote  the  ratio  of  f_  to  f+:  R  =  j^-.  In  this  section  we 
predict  the  performance  of  the  three  planning  algorithms 
based  on  the  value  of  R. 

Case  1:  R  C  1 

Since  SNLP  resolves  all  positive  threats,  it  imposes 
more  constraints  on  a  plan.  Thus,  on  the  average  an 
SNLP  plan  is  more  linearly  ordered  than  either  a  TWEAK 
plan  or  a  nonlin  plan.  A  more  linearly  ordered  plan 
has  a  smaller  number  of  existing  establishing  operators 
for  a  given  precondition,  and  thus  a  smaller  branching 
factor.  Thus,  the  branching  factor  of  SNLP  is  likely  to  be 
the  smallest  among  the  three,  and  that  for  tweak  is  the 
largest  due  to  its  conservative  stand  in  resolving  threats. 

When  t+  is  relatively  large,  the  total  number  of 
threats  t  resolved  by  snlp  is  large,  which  in  turn  in¬ 
creases  snlp’s  search  depth.  Also,  for  both  nonlin  and 
SNLP,  a  causal  link  has  to  be  built  for  every  precondition 
in  a  plan,  a  behavior  that  fixes  a  lower  bound  on  their 
search  depths.  With  many  positive  threats  in  a  plan,  a 


precondition  is  more  likely  to  be  achievable  by  an  ex¬ 
isting  step.  Therefore  TWEAK  will  be  able  to  skip  many 
more  preconditions  compared  to  nonlin  and  snlp.  Thus 
the  search  depth  of  TWEAK  will  be  much  less  than  both 
NONLIN  and  snlp,  and  the  search  depth  of  nonlin  will 
be  smaller  than  SNLP  because  it  does  not  resolve  positive 
threats. 

As  R  decreases  below  one,  the  branching  factor  for 
TWEAK  and  NONLIN  increase,  while  the  search  depth  for 
SNLP  increases.  The  time  complexity  foT  the  former  go 
up  polynomially,  while  for  the  latter  it  goes  up  exponen¬ 
tially.  Moreover,  the  depth  of  NONLIN  is  greater  than 
the  depth  of  tweak.  Therefore,  we  predict  that  when 
R  1  TWEAK  will  perform  better  than  nonlin,  which 
in  turn  will  perform  better  than  snlp. 

Case  2:  R  ss  1 

As  with  the  previous  case,  the  additional  constraints 
imposed  by  snlp  and  nonlin  over  TWEAK  imply  that 
SNLP  will  have  a  smaller  branching  factor  then  NON¬ 
LIN,  and  nonlin  will  have  a  smaller  branching  factor 
than  tweak.  However,  the  difference  in  the  number  of 
threats  t  resolved  by  TWEAK,  SNLP,  and  nonlin  will  be 
reduced  since  there  are  fewer  positive  threats  and  more 
negative  threats.  The  reduced  number  of  positive  threats 
will  reduce  the  depth  for  snlp  and  nonlin  and  the  in¬ 
creased  number  of  negative  threats  increases  the  chance 
that  TWEAK  will  be  forced  to  revisit  the  same  precon¬ 
dition/step  pair.  As  a  result,  the  performance  of  the 
different  planners  could  be  very  close  and  will  depend 
on  depth  and  branching  factors  for  the  problems  being 
solved. 

Case  S:  R  >  1 

TWEAK  is  likely  to  have  the  largest  branching  factor 
because  every  time  a  negative  threat  occurs,  all  existing 
and  new  operators  are  considered  as  establishes  again. 
This  effect  increases  the  factor  bjold  for  TWEAK,  result¬ 
ing  in  the  effective  branching  factor  for  tweak  being 
greater  than  both  SNLP  and  nonlin.  Also  due  to  its 
resolution  of  positive  threats,  a  SNLP  plan  is  likely  to  be 
more  linearised  than  a  nonlin  plan,  thus  the  branching 
factor  of  SNLP  will  be  smaller  than  NONLIN. 

Each  negative  threat  creates  a  chance  for  tweak  to 
revisit  the  same  precondition/step  pair.  Since  in  the 
R  ^  1  case,  there  is  a  large  number  of  negative  threats, 
the  number  of  times  each  precondition  is  visited,  vtweaki 
is  likely  to  increase.  Since  TWEAK  is  expected  to  have 
a  larger  branching  factor  and  depth  greater  than  both 
snlp  and  nonlin,  when  R  »  1  tweak  is  expected  to 
perform  the  worst,  snlp  will  outperform  NONLIN  slightly 
due  its  smaller  branching  factor. 

4.2  Empirical  Results 

In  order  to  verify  our  predictions  by  comparing  SNLP, 
nonlin  and  tweak  on  problems  with  different  ratios  of 
negative  and  positive  threats,  we  constructed  an  artificial 
domain  where  we  could  control  the  value  of  R.  In  this 
domain,  each  goal  can  be  achieved  by  a  subplan  of  two 
steps  in  a  linear  sequence.  Each  step  either  achieves  a 
goal  condition  or  a  precondition  of  a  later  step.  The 
preconditions  of  the  first  step  always  hold  in  the  initial 


state.  In  addition,  we  also  added  extra  operator  effects  to 
create  threats  in  planning.  The  difficulty  of  the  problems 
in  this  domain  can  be  increased  by  increasing  the  number 
of  goal  conditions  and  the  total  number  of  threat; 

(defstep  :action  A.i  :precond  /»  '.equals  {} 

:add  {P+-,  4+1  if  i  <  n+;  70  if  i  —  n  —  l  and  n+  >  0} 

:delete  if  0  <  i  <  n_; /n_j  if  i  —  0 

and  n_  >  0}' 

(defstep  .action  A,j  :precond  Pi  :equals  {} 

:add  {G*;  P,+j  if  i  <  n+;  P0  if  i  ~  n  —  1  and  n+  >  0} 

:delete  {Pj_i,  */  0  <  t  <  n_;  P„-t  */  *  =  0 
and  n_  >  0}) 

We  used  this  artificial  domain  to  run  a  set  of  experi¬ 
ments  to  compare  the  performance  of  the  different  plan¬ 
ners.  In  these  experiments  we  simultaneously  varied  the 
number  of  positive  and  negative  interactions,  such  that 
the  total  number  of  interactions  remained  the  same,  but 
the  ratio  R  changes  from  zero  to  infinity;  the  number 
of  negative  interactions  increased  from  0  to  9  while  the 
number  of  positive  interactions  decreased  from  9  to  0. 
Below,  we  present  the  results  of  our  empirical  tests  on 
different  points  of  the  spectrum  of  as  defined  by  the  ratio 
R. 

In  the  experiments,  each  problem  was  run  in  SNLP 
[Barrett  and  Weld,  1992],  a  version  of  NONLIN  and  a 
version  of  TWEAK  that  were  modified  from  SNLP.  The 
problems  were  solved  using  a  best-first  search  on  the 
solution  size  in  order  to  fairly  compare  the  size  of  the 
problem  spaces  being  searched  by  each  system.  All  the 
problems  were  run  on  a  SUN  IPC  in  Lucid  Common  Lisp 
with  a  120  CPU  second  time  bound.  For  each  value  of 
ratio  R,  we  ran  the  systems  on  20  randomly  generated 
problems.  The  points  shown  in  the  graphs  below  are  an 
average  of  the  20  problems. 

4.2.1  Branching  Factor 

The  branching  factor  results  are  shown  in  Figure  1. 
Most  of  our  predictions  for  branching  factors  are  observ¬ 
able  in  the  figure.  For  example,  due  to  its  conservative 
stand  in  resolving  both  positive  and  negative  threats, 
SNLP  imposes  the  most  constraints  onto  a  plan,  and  as  a 
results  it  generally  has  the  lowest  branching  factor.  Also, 
as  the  number  of  negative  threats  increases,  which  con¬ 
strains  the  possible  plans,  the  branching  factor  decreases 
to  one. 

However,  there  are  a  few  surprises  shown  in  the  fig¬ 
ure.  When  R  1,  we  had  predicted  that  tweak  would 
have  a  larger  branching  factor  than  SNLP  and  would  be 
similar  to  NONLIN.  This  prediction  cannot  observed  from 
the  figure.  In  order  to  explain  this  effect  we  have  broken 
the  branching  factor  into  the  two  parts  described  in  the 
analysis,  the  establishment  branching  factor  and  the  de- 
clobbering  branching  factor,  which  are  combined  to  form 
the  overall  branching  factor.  These  graphs  are  shown  in 
Figures  2  and  3.  As  shown  in  the  graphs,  the  smaller 
than  expected  branching  factor  for  tweak  is  due  to  a 
smaller  than  expected  establishment  branching  factor. 
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Figure  1:  Comprison  of  the  Average  Branching  Factor  Figure  3:  Comprison  of  the  Average  Dedobbering 
of  each  of  the  Algorithms  Branching  Factor  of  each  of  the  Algorithms 
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Careful  analysis  of  the  data  shows  that  this  dis- 
crepency  with  the  predictions  is  due  to  the  assumption 
that  the  branching  factor  is  uniform  across  an  entire 
problem-solving  episode.  In  fact,  where  there  are  many 
positive  interactions,  tweak  quickly  narrows  in  on  a 
plan  and  reduces  the  establishment  branching  factor.  In 
contrast,  because  both  SNIP  and  NONLIN  build  explicit 
causal  links  and  resolve  more  threats  they  spend  more 
time  in  the  early  plan  formation  stage  when  the  branch¬ 
ing  factor  is  higher.  Thus  overall,  SNLP  and  NONLIN 
expand  a  larger  part  of  the  search  space  that  has  a 
large  branching  factor,  while  tweak  uses  it  ability  to 
exploit  positive  threats  to  rapidly  traverse  that  part  of 
the  search  space. 


4.2.2  Depth 

The  comparison  of  the  search  depths  is  shown  in  Fig¬ 
ure  4  and  they  axe  as  predicted.  The  only  apparent  dis¬ 
crepancy  is  that  the  difference  between  SNLP  and  TWEAK 
should  be  larger  when  R  <C  1.  However,  the  graph  is 
a  bit  misleading  in  this  case  because  it  includes  prob¬ 
lems  that  could  not  be  solved  within  the  time  bound  by 
NONLIN  and  snlp  and  so  it  underestimates  their  search  < 

depth. 
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Figure  4:  Comprison  of  the  Average  Depth  of  each  of 
the  Algorithms 

The  overall  search  depth  is  composed  of  a  number  of  ( 

factors  described  in  the  analysis,  which  includes  the  frac¬ 
tion  of  the  preconditions  considered,  the  average  number 
of  times  each  precondition  is  visited, a  and  the  average 
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number  of  threats  detected  by  each  algorithm.  Figure  5 
shows  the  fraction  of  preconditions  considered.  This 
number  should  be  one  for  both  snlp  and  nonlin  but 
again  the  graphs  are  distorted  by  the  fact  that  these  two 
systems  did  not  complete  ail  of  the  problems  within  the 
time  limit.  In  that  case,  there  ar.  .  number  of  precon¬ 
ditions  of  operators  that  had  >•  t  yet  been  considered. 
Note  that  for  most  of  the  roblems,  tweak  only  ex¬ 
panded  roughly  60-80%  of  the  preconditions  and  as  the 
problems  had  fewer  positive  interactions,  it  was  forced 
to  expand  more  and  more  of  the  preconditions. 
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R  =  (negative  interactions  /  positive  interactions) 


Figure  5:  Comprison  of  the  Average  Fraction  of  Precon¬ 
ditions  Considered  by  each  of  the  Algorithms 


Figure  6  shows  the  average  of  number  of  times  each 
precondition  is  visited.  As  predicted,  snlp  and  nonlin 
visit  every  precondition  exactly  once,  while  tweak  vis¬ 
its  some  preconditions  more  than  once.  As  the  number 
of  negative  interactions  increase,  the  value  for  TWEAK 
increases  because  it  does  not  protect  the  conditions  that 
have  already  been  achieved. 

Figure  7  shows  the  average  number  of  threats  detected 
by  each  of  the  systems.  The  fact  that  SNLP  detects  a 
much  larger  number  of  threats  than  both  nonlin  and 
TWEAK  comes  as  no  surprise.  However,  the  fact  that  the 
number  of  threats  detected  by  NONLIN  is  less  than  the 
number  detected  by  TWEAK  when  <  1  was  not  pre¬ 
dicted  by  the  analysis.  This  appears  to  be  due  to  the  fact 
that  the  negative  threats  that  NONLIN  protects  against 
impose  additional  ordering  constraints  on  the  plan  and 
a  more  linearly  ordered  plan  has  fewer  potential  threats. 

4.2.3  Average  CPU  Time 

The  average  CPU  time  for  solving  problems  in  the 
artificial  domain  is  shown  in  Figure  8.  The  result  fits 
exactly  with  our  predictions.  One  thing  to  note  is  that 
no  system  performs  absolutely  the  best  throughout  the 
entire  spectrum  defined  by  R.  Another  is  that  although 
NONLIN  did  well  as  compared  to  SNLP  when  jR  is  small,  it 
is  never  significantly  better  than  SNLP.  In  the  case  where 
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R  =  (negative  interactions  /  positive  interactions) 


Figure  6:  Comprison  of  the  Average  Number  of  Times 
each  Precondition  is  Visited  by  each  of  the  Algorithms 


R  =  (negative  interactions  /  positive  interactions) 

Figure  7:  Comprison  of  the  Average  Number  of  Threats 
Detected  by  each  of  the  Algorithms 


it  does  outperfrom  SNLP  it  is  dramatically  worse  than 
TWEAK.  This  effect  should  lend  credibility  to  the  protec¬ 
tion  against  positive  threats  as  used  in  SNLP.  Although 
protection  of  positive  threats  seemed  clumsy  when  R  is 
small,  when  the  number  of  negative  threats  is  relatively 
large  the  protection  method  used  by  SNLP  imposes  more 
constraints  on  a  plan.  The  resulting  plans  in  SNLP’s 
search  space  are  more  linear  due  to  the  additional  con¬ 
straints.  The  computational  advantage  of  dealing  with 
a  more  linear  plan  compansates  for  the  loss  of  efficiency 
due  to  the  protection  of  positive  threats. 
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Figure  8:  Comprison  of  the  Average  CPU  Time  of  each 
of  the  Algorithms 

5  Related  Work  and  Conclusions 

As  we  stated  in  the  introduction,  little  work  has  been 
done  on  comparing  different  partial  order  planners.  An 
exception  is  the  work  by  Kambhampati[Kambhampati, 
1993;  Kambhampati,  1992],  who  (concurrently  with  our 
work)  carried  out  a  set  of  experiments  to  test  the  merits 
of  different  partial-order  planners.  In  that  work,  a  pair  of 
partial-order  planners  MP  and  MP-I  are  proposed  that 
build  upon  SNLP  and  nonlin  by  making  use  of  multiple 
contributors  to  achieve  a  precondition.  Experiments  in 
a  set  of  closely  related  domains  were  conducted,  and  the 
resulting  comparison  of  snlp,  nonlin,  tweak,  MP,  and 
MP-I  show  that  MP-I  outperforms  all  of  the  rest,  and 
that  NONLIN  in  one  test  performed  much  better  than 
both  snlp  and  TWEAK(Figure  8,  [Kambhampati,  1993]). 

Contrasting  Kambhampati’s  results  to  ours,  we  note 
that  the  former  is  based  on  a  fixed  domain.  Our  results 
clearly  demonstrate  that  varying  the  ratio  R  of  positive 
to  negative  threats  experienced  by  a  planner,  almost  any 
comparison  result  can  be  obtained;  when  5  <  1  the 
comparison  results  should  be  dramatically  different  from 
that  when  R  1.  Thus,  it  is  not  surprising  that  one 
can  find  a  domain,  with  a  specific  R  value,  where  SNLP 
and/or  TWEAK  perform  worse  than  NONLIN.  From  this 
perspective,  the  work  by  Kambhampati  can  be  seen  as 
orthogonal  to  ours;  while  we  search  for  domain  features 
by  which  to  determine  the  relative  performance  of  each 
system,  Kambhampati  looks  for  the  best  planner  on  a 
single  point  in  the  spectrum  of  features. 

In  summary,  we  have  presented  a  detailed  compari¬ 
son  of  the  goal  protection  strategies  used  in  the  snlp, 
NONLIN,  and  TWEAK  planning  algorithms.  The  analy¬ 
sis  provides  a  foundation  for  predicting  the  conditions 
under  which  different  planning  algorithms  will  perform 
well.  As  the  results  show,  snlp  and  nonlin  performs 
better  than  tweak  when  the  ratio  of  negative  threats 
to  positive  threats  is  large,  and  TWEAK  performs  signifi¬ 


cantly  better  than  SNLP  and  NONLIN  in  the  opposite  case. 
The  implications  of  these  results  for  someone  building  a 
practical  planning  system  is  that  the  most  appropriate 
goal  protection  strategy  depends  on  the  characteristics 
of  the  problem  being  solved.  This  paper  provides  an  im¬ 
portant  step  in  building  useful  planners  by  identifying  a 
feature  of  planning  domains  that  has  a  major  impact  on 
the  performance  of  different  planning  algorithms. 
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